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The  Spring  Symposium  on  Knowledge-based  Environments  for  Teaching  and  Learning  focused 
on  the  use  of  technology  to  facilitate  learning,  training,  teaching,  counseling,  coaxing  and 
coaching.  Sixty  participants  from  academia  and  industry  assessed  progress  made  to  date  and 
speculated  on  new  tools  for  building  second  generation  systems. 

Selection  of  topics  and  participants  was  motivated  by  a  desire  for  ideological  breadth  and  depth. 
Panel  leaders  included  William  J.  Clancey  and  Alan  Lesgold  (researchers  of  real-world  systems); 
Kurt  VanLehn  (champion  of  cognitive  models);  Beverly  Woolf  (defender  of  discourse  systems); 
Elliot  Soloway  (advocate  for  alternative  environments);  and  Sarah  Douglas  (spokesperson  for 
supportive  systems). 


Human-Computer  Interaction 


Researchers  have  moved  away  from  building  omniscient  tutors  capable  of  detecting  all  possible 
errors  and  misconceptions.  Instead,  research  is  now  focused  on  building  empathetic  partners 
t-hai.  Uiucse  from  among  several  forms  of  interaction  based  on  the  content  of  the  communication 
and  the  needs  of  the  stuueni  (Woolf,  1988].  Possible  communication  styles  include  didactic 

‘Beverly  Woolf  and  Dan  Suthers  have  received  partial  support  from  the  National  Science  Foundation  under 
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2Kurt  VanLehn’s  research  is  supported  by  the  Office  of  Naval  Research  under  contracts  Nonoia.ps-K-OC"'* 
and  N00014-86-K-061  6. 
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explanation,  guided  discovery  learning,  coaching  or  coaxing,  and  critiquing.  Although  no  one 
style  is  preferred,  different  tutorial  applications  will  be  better  addressed  with  a  given  primary 
style. 

For  example,  as  explained  by  Dan  Suthers  and  James  Lester,  didactic  explanation  is  good  for 
communicating  a  body  of  declarative  knowledge  shared  by  some  community  (e.g.  biologists). 
In  such  applications,  the  student  needs  to  learn  the  community’s  terminology,  and  thus  didactic 
explanation  may  be  more  efficient  than  requiring  that  the  student  rediscover  the  principles  of 
the  field  on  his  or  her  own.  On  the  other  hand,  the  more  active  nature  of  discovery  learning  helps 
the  student  “own”  the  acquired  knowledge  to  a  greater  extent  than  can  didactic  explanation. 

The  style  of  interaction  varies  within  a  tutorial  domain  as  well  as  across  types  of  domains.  For 
example,  Lewis  et  al.  [1990;  showed  how  a  (human)  guided  discovery  tutor  changed  strategies 
from  script-like  to  opportunistic  when  students  suggested  am  activity  or  showed  the  need  for 
remediation  of  a  deficiency. 


Communication  Research  Issues 


Pressing  research  issues  in  human-computer  communication  were  identified  both  in  artificial 
intelligence  and  in  education.  In  artificial  intelligence ,  research  issues  include  the  representa¬ 
tion  and  control  of  knowledge.  From  this  perspective,  knowledge  of  didactic  explanation  might 
be  represented  and  organized  in  a  system,  along  wbth  the  basic  knowledge  of  a  domain.  In¬ 
dexing  mechanisms  for  accessing  different  perspectives  on  the  topic  should  be  designed  using 
abstractions  appropriate  for  the  content  selection  task. 

Choosing  and  organizing  domain  knowledge  provides  the  next  set  of  research  issues.  Control 
should  account  for  the  tutor’s  ability  to  dynamically  switch  strategies  according  to  multiple 
constraints  in  a  manner  be  sensitive  to  features  that  human  tutors  use  in  tutorial  interactions. 
The  tutor  should  consider  available  student  modeling/diagnosis  when  making  tutorial  decisions 
based  on  multiple  goals.  Further  work  is  required  in  characterizing  “relevance”  for  selecting 
knowledge  for  didactic  explanation,  especially  when  multiple  perspectives  on  the  topic  are 
available.  Even  when  the  primary  emphasis  is  on  stimulating  the  student’s  own  creativity  and 
intelligence,  the  program  s  design  must  still  be  based  on  solid  theory  of  relevance  to  select  its 
actions  and  response.  To  do  so,  memory  and  pragmatic  knowledge  should  be  brought  to  bear 
on  language  processing. 


Another  research  issue  concerns  the  characterization  of  coherence  in  machine  response.  Is 
coherence  a  property  of  the  “knowledge  pool”  to  be  used  in  generating  the  next  response  or  a 
property  of  the  dialogue  or  both?  In  choosing  content  from  a  multiple  granularity  knowledge 
base,  how  do  we  ensure  that  the  chosen  pool  of  knowledge  is  coherent  given  the  dialogue  context? 

Educational  research  issues  focus  on  adequately  modeling  the  student  and  the  pedagogical 
context  (see  next  section),  and  then  recognizing  how'  a  system  might  stimulate  and  facilitate 
the  student’s  own  abilities  and  creativity. 
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A  separate  issue  concerns  how  relevant  knowledge  should  be  presented  once  it  has  been  selected. 
For  example,  the  tutor  might  state  generalizations,  use  case  examples,  or  provide  analogies. 
Presentations,  whether  explanations  or  examples,  must  be  presented  in  such  a  way  that  the 
student  will  be  prepared  to  understand  new  material  and  integrate  it  into  an  existing  conceptual 
framework,  or  into  one  which  has  been  built  up  in  the  preceding  dialogue.  We  need  to  better 
understand  how  to  choose  and  coordinate  multi-media/modality  presentations  at  the  interface 
media  level,  e.g.  the  use  of  text,  diagrams,  charts,  pictures,  animation,  and  sound. 

In  summary,  despite  much  work  attempting  to  do  so,  we  still  need  to  figure  out  how  to  make 
dialogue  sensitive  to  dialogue  context  and  to  what  is  known  or  knowable  about  the  student's 
‘"state”. 


Cognitive  Modeling 


The  cognitive  modeling  group  provided  strong  advocacy  for  cognitive  modeling  at  the  sympo¬ 
sium.  They  argued  for  increased  use  of  modeling  at  three  stages  of  design  of  knowledge-based 
systems,  primarily  (1)  development  of  pedagogical  and  subject-matter  theories,  (2)  design  of 
instruction,  and  (3)  delivery  of  instruction.  Of  these  phases,  the  design  of  instruction  is  the 
one  that  seems  to  have  achieved  the  most  direct  benefit  from  cognitive  modeling,  including 
substantial  benefits  from  modeling  subject  matter  experts.  For  instance,  Anderson  et  al.  [1990[ 
attribute  much  of  the  success  of  their  tutors  to  the  cognitive  task  analysis  of  experts  in  Lisp, 
geometry  and  algebra. 

Work  on  modeling  good  teachers  and  tutors  has  only  just  begun  (with  the  exception  of  a  few 
early  classics,  such  as  the  work  of  Stevens  and  Collins  on  Socratic  tutoring  [1977’).  YanLehn 
expects  this  line  of  investigation  to  pay  off  at  least  as  well,  if  not  better,  than  the  modeling  of 
experts  and  learners. 

Of  the  three  phases  of  pedagogical  work,  the  actual  delivery  of  the  instruction  is  the  area  where 
cognitive  modeling  has  found  the  least  fruitful  application.  Mostly,  this  is  due  to  a  historical 
accident.  In  most  systems  to  date,  teacher  models  have  been  weaker  than  expert  models  and 
student  models.  Although  a  good  teacher  model  might  compensate  for  an  impoverished  expert 
or  student  model,  experience  has  showm  that  strong  expert  and  student  models  require  a  decent 
teacher  model  for  the  system  to  be  effective. 

YanLehn  underscores  the  fact  that  modeling  is  just  good  engineering  practice,  regardless  of 
whether  one  is  building  a  hydroelectric  dam  or  a  science  course.  Writh  tongue  in  cheek,  he 
su6gests  that  if  students  could  sue  malfeasant  instructional  developers,  cognitive  modeling 
would  be  much  more  common  since  it  is  so  obviously  effective. 

William  J.  Clancey,  however,  was  more  reserved  about  the  utility  of  cognitive  modeling.  While 
acknowledging  that  building  such  models  is  possible,  he  questions  the  relation  they  have  to 
mechanisms  of  human  learning.  For  instance,  does  the  model  show  the  student  how  to  interpret 
and  generate  domain  concepts,  or  does  it  simply  justify  the  machine’s  presentations?  Clancey 
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would  Like  to  see  alternative  cognitive  models  available  within  a  system  rather  than  a  single 
‘correct’  model  used  to  justify  instruction. 


Understanding  Plans  and  Goals 


In  the  move  away  from  building  all-knowing  and  all-powerful  tutors,  researchers  have  focused 
on  developing  environments  that  implicitly  elicit  information  about  student  goals  and  plans. 
Human  dialogue  succeeds  despite  ambiguity  and  digressions  because  both  participants  model 
the  discourse,  the  subject  matter,  and  die  other  speaker;  and  both  participants  actively  work 
towards  success  of  the  discourse. 

This  suggests  that  continuing  efforts  be  made  to  enhance  the  machine’s  ability  to  do  its  part. 
Techniques  such  as  plan  recognition  and  learning  still  play  only  a  small  role  in  current  teaching 
systems.  Interfaces  were  described  that  inquire  about  beliefs  and  high-level  thoughts  while 
supporting  meta-cognitive  activities.  Students  might  choose  from  a  menu  of  high-level  plans, 
such  as  a  menu  item  in  an  Algebra  tutor  that  says  “collect  all  variables  to  one  side  of  the 
equation.”  Such  interfaces  require  more  careful  analysis  and  structuring  of  the  task  domain  and 
of  cognitive  structures;  they  also  require  mechanisms  to  support  co-operative  dialogue  and  tc 
‘understand’  student  perspectives. 


Real-World  Applications 


William  J.  Clancey  and  Alan  Lesgold  led  several  discussions  on  the  impact  of  knowledge- 
based  systems  in  industry  and  the  military.  The  clear  emergence  of  new  architectures  and 
positive  training  results  have  produced  the  feeling  that  progress  is  being  made.  Indeed,  several 
systems  were  described  which  achieve  the  two-sigma  effect  [Bloom,  1984],  which  is  the  same 
improvement  in  learning  that  results  from  one-on-one  human  tutoring  over  classroom  tutoring. 
Several  success  stories  were  described  in  which  students  using  tutors  learned  knowledge  and 
skills  in  one-third  to  one-half  the  time  it  took  for  a  control  group  to  learn  the  same  material 
[Shute,  1990]. 

In  one  special  case,  students  working  with  an  Air  Force  electronics  troubleshooting  tutor  for  only 
20  hours  gained  a  proficiency  equivalent  to  that  of  trainees  with  40  months  (almost  4  years)  on- 
the-job  training  [Lesgold,  Lajoie,  Bunzo  &  Eggan,  1990].  In  another  example,  students  using  a 
LISP  tutor  at  Carnegie- Mellon  University  [Anderson,  1990]  completed  programming  exercises  in 
30time  than  those  receiving  traditional  classroom  instruction  and  scored  43using  a  a  microworld 
environment  learned  general  scientific  inquiry  skills  and  principles  of  basic  economics  in  one-half 
the  time  required  by  students  in  a  classroom  setting  [Shute,  Glaser  &  Raghavan,  1989]. 

Given  these  results,  the  group  asked  why  more  tutors  were  not  being  used  and  why  existing 
systems  were  not  more  effective.  One  reason  why  industry  and  the  military  have  not  widely 
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adopted  these  systems  relates  to  the  lack  of  artificial  intelligence  development  tools.  such  as 
shells  and  frameworks,  similar  to  the  shells  used  to  build  rapidly  expert  systems.  Tools  would 
facilitate  large-scale  development;  and  a  simple  tool,  such  as  a  simulation  tied  to  an  expert 
system  or  to  a  lock -step  tutor,  might  be  a  practical  way  for  a  designer  to  get  started  on  a 
path  of  incremental  design  through  feedback  from  the  user.  Some  researchers  suggested  that  a 
teacher  should  interact  with  a  variety  of  tools,  much  as  a  director  might  orchestrate  a  suite  cf 
tools. 

Other  reasons  for  the  slow  adoption  of  new  systems  might  include  the  need  to  reduce  cognitive 
task  analysis  to  engineering  practice  and  the  need  to  make  widely  available  knowledge  repre¬ 
sentations  (i.e.  qualitative  simulations)  which  are  better  than  those  offered  by  first-generation 
expert  system  tools.  An  additional  barrier  is  the  lengthy  development  cycle  required  before  a 
system  can  move  from  research  lab  to  a  saleable  product. 


‘Hot’  Research  Issues 


Several  areas  emerged  as  ‘hot’  or  new  research  areas.  These  were  discussed  throughout  the 
symposium. 

Situated  learning  (and  teaching/acting/planning)  arose  frequently  as  a  topic.  It  was  espoused 
primarily  by  William  J.  Clancey,  Jeremy  Roschelle,  and  Etienne  Wenger  all  from  the  Institute 
for  Research  on  Learning,  Palo  Alto.  Since  situations  or  contexts  in  which  a  skill  is  learned  can 
not  be  exhaustively  or  completely  described,  training  systems  inevitably  predetermine  what  is 
relevant.  Similarly,  conventional  Artificial  Intelligence  models  of  expertise  leave  out  how  experts 
know  what  is  relevant  and  how  they  change  their  minds.  This  approach  suggests  that  Artificial 
Intelligence  systems  need  to  place  increasing  emphasis  on  representation  as  an  activity  within  a 
perceptual  space  and  organized  by  social  interaction.  Current  systems  omit  the  social  context  in 
which  domain  representations  are  created,  justified,  and  changed.  At  present,  knowledge-based 
cognitive  modeling  cannot  characterize  the  work  somebody  must  do  to  understand  specific 
artifacts  or  tools  of  a  community.  One  reason  why  on-the-job  training  is  more  cost  efficient  is 
because  there  is  no  need  to  simulate  what  can’t  be  made  fully  explicit  anyway. 

Computer  as  mediator  -  Jeremy  Roschelle  demonstrated  that  a  system  could  facilitate 
discussion  amongst  several  students  and  could  support  their  own  explanations  to  each  other. 
In  such  a  case,  the  computer  becomes  a  mediator  empowering  both  students  and  teachers. 

Andrea  diSessa  showed  that  the  goals/cap  abilities  to  be  taught  are  negotiable;  he  enables 
students  to  discover  their  own  interests  and  assists  them  in  pursuing  same.  For  example, 
he  showed  a  Boxer  system  which  supports  young  students  in  inventing  representations  using 
graphic  tools.  In  other  words,  they  discovered  the  rules  of  graph  construction. 

Empowering  curriculum  designers  -  Jim  Spohrer  described  systems  developed  at  Apple 
to  assist  curriculum  designers  in  the  use  of  multi-media.  Oliver  Selfridge  challenged  the  group 
to  question  the  nature  of  the  learning  task  implicit  in  their  teaching  machines. 
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Qualitative  reasoning  -  Ken  Forbus  demonstrated  that  a  system  could  qualitively  modei  a 
complex  domain,  e.g.  a  steam  boiler  or  a  propulsion  plant,  an^  *hat  this  representation  could 
be  used  for  teaching.  His  work  on  qualitative  modeling  is  now  10  years  old  and  is  approaching 
the  pou  t  of  formalizing  the  reasoning  needed  in  qualitative  modeling. 


Conclusions 


Given  the  diverse  backgrounds  and  methodology  of  participants,  little  commonality  could  have 
been  expected.  However,  a  small  consensus  was  achieved  and  some  new  scientific  ground  bro¬ 
ken.  Agreement  was  reached  on  the  need  for  a  variety  of  discourse  approaches,  and  the  need 
for  cognitive  models,  although  no  single  solution  to  achieve  widespread  use  of  either  was  forth¬ 
coming. 

Several  areas  require  further  research.  Basic  research  is  needed  in  planning  and  plan  recogni¬ 
tion.  building  natural-language  interfaces,  and  applying  architectures,  such  as  blackboards  to 
teaching  systems. 

From  the  viewpoint  of  communication,  the  symposium  was  a  real  success;  discussion  was  lively 
and  at  times  controversial.  Research  appears  to  be  strong  in  depth,  broad  in  perspective, 
and  motivated  by  the  promise  of  building  more  powerful  teaching  environments  with  greater 
knowledge,  increased  inference  capability,  and  more  complex  reasonmg  ability.  Researchers  are 
very  active  and  the  field  seems  to  be  alive  and  well. 
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